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Cockpits in the Era of Breakthrough Flight 


Introduction 


Responding to the request to explore forefront science relevant to future cockpits for 
any form of aerospace craft and/or deep-space craft that is propelled by any 
unspecified advanced or breakthrough propulsion physics, this report offers a 
provisional cockpit design that employs the following: 


e Predictions of propulsion physics breakthroughs. 
e Lessons of human-machine interface. 


e Emerging technology for displays and controls. 


This study discusses the implications of breakthrough propulsion, including the mastery 
over gravitational and inertial forces and the prospect for faster-than-light spaceflight. 
The main reference used to predict these possibilities is the book Frontiers of Propulsion 
Science [Millis & Davis, 2009]. Although the breakthroughs discussed in this book are 
not imminent, enough progress has been made to allow for thoughtful speculation 
about their characteristics and possible implementations. 


How these advances may affect future cockpits is described, and this is the central 
message of this study. The most significant differences from legacy cockpits are 
identified and then used to set the baseline design requirements. 


Additionally, substantial lessons about human-machine interfaces are reviewed and 
applied to this notional cockpit. Most of this progress relies on better accommodating 
the norms and limitations of human perception—lessons that do not change even when 
vehicle characteristics change. 


Recent advancements in the use of hand gestures for commands are also included, as 
well as advancements in brain-machine interfaces. In this conceptual study of far-future 
possibilities, these technologies are assumed to have reached full maturity, with one 
exception: in order to focus this study on future cockpits, the options for brain implants 
and for transhumanism—where humans are reengineered to adapt to new 
requirements—are not considered. 


Next-step investigations are suggested to refine the ideas presented herein. A caveat 
is that advances in cockpits for breakthrough flight might be further advanced by taking 
advantage of the gaming industry techniques or through science fiction speculation. 


Note: All projections in this report are based on public domain information. 
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Chapter 1: Predicting Implications of Propulsion 
Breakthroughs 


MARCH OF PROGRESS: REVOLUTIONARY PROPULSION PHYSICS 


Breakthroughs in propulsion physics (such as the control over gravitational or inertial 
forces, propellant-less space drives, and even faster-than-light travel) are not 
imminent; however, enough progress has been made to allow for thoughtful 
speculation about their nature and implications. As a preview, the implications to 
cockpit design include added degrees of motion, combination of operational regimes 
(near ground, orbit, and beyond), greater range of speed (from zero-speed hover to 
beyond light speed), and loss of familiar motion cues (pilot’s inertia and visual cues) 
resulting from the separation of external and internal environments. 


The primary reference used to predict these possibilities is the book Frontiers of 
Propulsion Science [Millis & Davis, 2009],! particularly chapters 3, 4, and 15. This book 
may be the first-ever scholarly compilation of science pertaining to breakthrough 
flight—methods sufficiently advanced to enable human voyages to other star systems. 
The book examines a wide range of works, offering introductory explanations and 
comparisons between approaches and identifying high-priority unknowns needing 
deeper study. References to specific ideas and issues cite that book and other original 
works. 


Setting Ideal Performance as Design Target 


This report focuses on the most significant likely differences between contemporary 
cockpits and cockpits in the era of breakthrough flight. Possibilities that imply the most 
demanding changes are considered first, and explanations of the correlations between 
the propulsion characteristics and resulting cockpit features are provided. Looking to 
the far future, this study evaluates the impact of having achieved the following 
breakthrough advancements: 


* Control over gravitational and inertial forces: 


- The craft is propelled by interacting with the properties of the space-time and/or 
inertial frames surrounding the craft—and can accelerate at g levels beyond 
human endurance. 


- The environment inside a craft can be sustained anywhere between 0 g andi1g 
(minimum range) without regard for either the motion of the craft or its outside 
gravitational environment. 


e Faster-than-light (FTL) speeds are possible by having mastered control over those 
aspects of nature that impose the light-speed limit. However, due to reasonable 
relativistic projections of the energy required for propulsion coupled with the limits 
of the human lifespan, it is reasonable to expect that travels will be limited to within 
our galaxy. For the sake of bracketing the scope of coverage, this study assumes 
that practical star flight will be limited to a 100-light-year radius around our Sun. 
Even with this constraint, thousands of star systems are within that range. 
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* The energy supply for these features resides on the vehicle and is considered to 
have a dynamic interplay with the motion of the vehicle. The energy can be 
transferred to and from the environment surrounding the craft as a consequence of 
the propulsive maneuvers. 


Sanity Check on Predictions 


Objectively, the propulsion physics predictions offered in this report should be 
interpreted as informed conjectures or, at best, well-reasoned speculations. Absent of 
verified theories and engineering implementations, it is premature to consider this first 
study as the last word on this topic. Further progress will likely reduce the span of 
options and provide greater insight into implementation details. 


It must also be stressed that these interpretive predictions and cockpit implications are 
solely generated by the author and, thus, have not yet been published or debated with 
other scientists and engineers. Therefore, the reader should consider these predictions 
to be an initial step into the process. 


VEHICLE AND COCKPIT IMPLICATIONS 


Ideally, it is desirable to have a vehicle that can move in any direction, at any speed, in 
both air and space, without limitations. These features imply the need to have 
technological mastery over the forces of gravity and inertia and mastery over those 
aspects of nature that impose the light-speed limit. Based on projections of the 
underlying physics, such abilities would have secondary characteristics that affect how 
such motions are monitored and controlled. 


Degrees of Freedom 


Unlike an aircraft, whose motion consists basically of deviations from constant forward 
motion, or a helicopter, whose motion is dominated by the dynamics of its main rotors, 
a breakthrough propulsion vehicle would allow the full six degrees of freedom, including 
the ability to remain fixed relative to a desired reference. For example, if we start with 
the situation of a vehicle hovering over the ground, the breakthrough vehicle should be - 
able to change its orientation (yaw, pitch, or-roll) without affecting its altitude or lateral 
position. Similarly, it should be able move up/down or betel without the need to 
induce pitch or roll maneuvers (Figure 1). 


Such novel motion leads to two major differences from legacy cockpits: 


e Independent control inputs are needed for the full six degrees of freedom (yaw, 
pitch, and rall; and laterally, x [fore-aft], y [left-right], and z [up-down]). 


e New display methods are required to convey position, orientation, and motion for all 
those degrees of freedam. 


The control methods need not copy legacy methods from airplanes or helicopters— 
methods that are based on the mechanisms of their origin (Figure 2). Instead, future 
cockpit designs are now free to use control methods tailored to the natural 
action/reaction of pilots, while the vehicle's interfaces perform the function of 
converting pilot inputs to drive the vehicle's motion. Whether such a system consists of 
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a single joystick with six degrees of freedom, some sort of gesture-based system, or 
one that has those degrees of freedom dispersed across multiple pilot inputs (@.g., head 
motion, legs and feet, and arms and hands) remains open for future study. As a 
provisional baseline, this report chooses the option of having a pair of six-degree-of- 
freedom joysticks, one for both the left and right hands and located at the edge of the 
cockpit chair's arm rests. 
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Figure 1. Six Independent Degrees of Freedom. (Graphic: A. Szames] Note: the vehicle shown is strictly 
hypothetical and is a combination of three 1960s science fiction vehicles: Seaview submarine, Galileo shuttle, and 
Amtronic car. 


Similar to requiring new control methods, new display methods are also required to 
convey more information than in legacy cockpits. In addition to the complete six 
degrees of freedom, these motions will take place near the Earth’s surface, in orbit, and 
in deep space. A key difference spanning those regimes is the traditional role played by 
a gravitational field as a reference for orientation and motion. Since a gravitational 
reference will not always be present, and yet is extremely important when it is present, 
the new display system must accommodate all regimes in a way that feels natural to 
the pilots. These particular challenges are addressed in the section on Mixed 
Operational Regimes. 


Separation of Internal and External Environments 


Probably the most significant and perplexing difference for breakthrough-era cockpits is 
that the sensations of motion inside the vehicle will not necessarily match the motion of 
the vehicle itself. This is both a consequence of the method of propulsion as well as 
cockpit features designed to ensure crew survival. 
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As illustrated in Figure 2, when planning for breakthrough flight, there is no need to 
constrain designs to match the legacy conventions derived from prior vehicles. In the 
case of both the airplane and the helicopter, the control inputs available to the pilot are 
specific to the mechanisms of the control surfaces, When projecting breakthough flight, 
it is assumed that the controls will be tailored to match the natural characteristics of 
pilots, and the propulsion system will be designed to perform accordingly. 
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Fig 2A. Airplane Motion 
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Figure 2. Comparing Conventions of Aircraft Motion. [Credit: A. Szames] 


Fig 28. Helicopter Motion 
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Fig. 2C, Ideal Full Motion 
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To make this easier to grasp and to provide a provisional concept, imagine that the 
vehicle is partitioned into concentric sections as shown in Figure 3. The central volume 
is for the crew, where it is required that the gravitational and inertial forces be 
sustained within survivable levels. The inner shell, or inner hull, surrounding that region 
contains whatever devices provide that safe environment. The outer shell contains the 
propulsion devices that induce the desired motion of the craft relative to the external 
space. The region between the two hull shells is a provisional separation for analyzing 
the interaction between those two functions. 


Coordinate system of the Crew cabin where inertial 
external environment and gravitational forces 
are maintained at normal, 
\ Jo ge Se safe conditions 
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Figure 3. Necessary Distinction Between External and Internal Ferce Environments. (Graphic: A. Szames] 


For analytical purposes, it is advantageous to consider these volumes and control 
surfaces (the hull shells) as separate and then use them to define boundary conditions. 
In addition to its utility for assessing future cockpit designs, this multisectioned vehicle 
baseline is valuable for gedanken experiments about revolutionary propulsion concepts. 


Why This Is Odd 


Conventionally, gravitational and inertial effects permeate through everything, so the 
notion of having different conditions inside and outside of the craft runs contrary to 
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experience. If the gravitational environment outside the craft is 1 g, that same 1 g is 
expected inside as well. Similarly for accelerations, the entire mass of the vehicle, 
including its occupants, will experience the same inertial reactions as the vehicle 
accelerates. 


Upon the advent of breakthrough propulsion, mastery over gravitational and inertial 
forces will have been achieved. This implies, for example, that it is possible for a vehicle 
to accelerate at extreme g's while its crew remains within survivable limits or that, 
while cruising in deep space (absent of any acceleration or gravitational field), the crew 
can enjoy the familiar, constant 1 g upward. 


Because such possibilities run so contrary to established experience, it is difficult to 
comprehend how such things can be achieved and then contemplate the consequences 
that these advances impose onto other systems of the vehicle—in this case, how they 
affect cockpit designs. Not all known approaches to propulsion physics evoke the need 
for a double hull.2 Versions that simply suggest a new thrusting mechanism and 
reaction mass would only need the double hull if also addressing how to provide a safe 
acceleration environment for the crew. As stated earlier, however, the most significant 
possible impacts are considered for this study. Therefore, two examples are described 
next for how breakthrough propulsion would require this provisional double-hull 
configuration. 


Why Double-Hull Needed: Example 1 (Warp Drive) 


The Alcubierre warp drive, which uses the physics from the Riemannian geometry of 
Einstein's general relativity, creates a "warp bubble" around the craft, and then this 
bubbie of space-time is moved through the surrounding space-time. This effect is 
created (in theory) by expanding space-time behind the vehicle and contracting space- 
time in front. The vehicle within the bubble feels no acceleration forces. As illustrated 
in Figure 4,3 the high peaks represent expanding space-time, while the low peaks 
represent contracting space-time. Note that the inner region is flat (meaning that the 
vehicle does not experience any acceleration forces}, that the region far from the 
propulsion effect is also flat, and that these two regions are separate. 
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Figure 4. Warp Drive. Taken directly from the Frontiers book, this image has become the iconic representation 
for a warp drive: the "York Extrinsic Time Plot.” 


In theory, the outer shell is presumed to create the propulsive effect of warping space- 
time outside the craft without affecting the inside. In short, the Alcubierre warp drive 
creates a separation of space-time environments inside and outside of the craft, 
although the exact details remain uncertain. 


Although there is no explicit function for the inner hull in this situation, this Alcubierre 
warp drive at least illustrates the concept of separation of these outer and inner 
environments. When planning future cockpits, the outer and inner inertial frames need 
to be treated as two distinct zones. The physics related to such considerations is still 
evolving? and beyond the scope of this report. 


Why Doubie Hull Needed: Example 2 (Field Space Drive) 


Another class of conceptual propulsion is a "field drive'—a subset of "space drives" 
where a spacecraft is propelled "...using only the interactions between the spacecraft 
and its surrounding space..."° Instead of using the Riemannian geometry of Einstein's 
general relativity, these approaches use the physics of fields and scalar potentials.® 
While several variants exist, the "Bias Drive" concept is selected here to illustrate the 
relevance of the double-hull configuration, specifically in the context of modifying the 
scalar potential that defines an inertial frame. By altering the properties of the 
surrounding inertial scalar, a gradient in that scalar is induced which, in turn, induces 
gravitational-like forces on matter located in that gradient. 
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Figure 5 represents one version of this effect,’ where the vehicle and its contents would 
be located at the steep gradient and therefore would jointly experience acceleration 
forces. Taken directly from the Frontiers book, this image represents what would 
happen if it were possible to asymmetrically modify Newton's gravitational constant to 
induce gravitational gradients that would then, in turn, accelerate the vehicle. Despite 
the similarity to the distortions in the warp drive (Figure 4), these surfaces represent 
scalar potentials of gravitational fields. Figure 6 is a modified version of this where the 
double-hull configuration of Figure 3 is imposed, along with the condition that the inner 
region remains unaffected. In Figure 6, the locations of the inner and outer shell are 
identified on the figure. The notion of the gravitational bias drive has been modified 
here to illustrate the concept of having two separate zones of inertial and gravitational 
forces. In this case, the plotted surface represents a scalar potential of an inertial 
frame, where a gradient has the same effect as a gravitational field. The smaller central 
flat area is the inside of the vehicle where no acceleration forces are felt. The outer 
edges of the diagram are also flat, representing the unaffected space sufficiently far 
fram the propulsive effect. The distorted regions in between represent the effects of 
both the outer and inner hull shells. The outer hull shell induces a gradient that propels 
the vehicle, and the inner shell acts to prevent those distortions from reaching the crew 
cabin. It should be emphasized that these notions are at the level of thought 
experiments, as opposed to being mature theory. 


a) Multiplicative mogification, B, Eq, (39a) bh) Exponential modification, 6, Eq. (39b) 


Figure 5. Hypothetical Gravitational Bias Drive. 


In short, this Inertia! Frame Bias Drive has affected the space both outside and inside of 
the craft to propel the vehicle and to keep its crew isolated from the resulting 
acceleration forces. For example, if the outer hull creates a 10-g field outside the craft, 
the inner hull would compensate to create an opposing field such that the crew cabin is 
free of the 10-g acceleration forces. It can, therefore, be speculated that—if such field- 
affecting physics is discovered—the inner shell could create a 1-g field when the craft is 
coasting in 0-g deep space. 
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Figure 6. Inertial Frame Bias Drive and Vehicle Zones. [Credit: M. Millis] 


To be explicit, the physics and engineering to create such situations do not yet exist. 
The related physics can be categorized as still being at steps one and two of the 
scientific method: defining the problern and collecting data.’ Among many other issues 
eluding discovery and resolution, major issues include momentum conservation, the 
role played by inertial frames, and methods to affect gravitational and inertial 
properties of matter and space. 


Secondary Consequences 


Pertinent to cockpit design, the normal sensations of motion inside the cockpit will likely 
not be the same. In contrast to the advantage of shielding the crew from harsh 
maneuvers, this shielding removes sensations of motion (seat-of-pants feeling) that 
pilots use to help judge the motion of their vehicle. This detriment is compounded by 
the likelihood of inducing motion sickness, since the visual cues of the vehicle's real 
motion will be different from that felt by the pilot. A difference between visual and 
vestibular cues is a cause of motion sickness. The option of allowing a certain portion of 
the vehicle's g-loading to be transferred to the cockpit can be considered as a 
mitigation strategy. Accordingly, cockpit controls to affect such changes are required. 


Also, it is likely that this double-hull notion would prevent direct visual contact between 
the occupants and the environment outside the vehicle, or perhaps distort such visual 
cues beyond easy interpretation. In other words, do not expect windows. Without the 
familiar visual and vestibular cues directly available to the pilot, it becomes vitally 
important for the cockpit displays to provide reliable and instinctive cues for the pilot to 
aptly judge the position, orientation, and motion of the vehicle. 


Mixed Operational Regimes 


A major desirable feature sought from propulsion breakthroughs is the ability to move 
from the surface of the Earth directly into space. This implies that the vehicle's displays 
and controls must readily encompass motion near the Earth’s surface, ascent into space, 
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transitions into and out of orbits, and long-duration sustained cruising in a O-g 
environment. 


As alluded to earlier, this deviates from prior displays where the Earth's gravitational 
field is available from which to gauge orientation. Similarly, the notion of an altimeter 
takes on a whole new meaning in this context. While visual cues for "up" are 
instinctively clear near the surface of the Earth (or even in closed rooms where 1 g is 
present), for a true breakthrough vehicle, these will be special conditions amongst a 
greater span of possibilities. 


A particular consequence of these added operational regimes is that unfamiliar 
situations are presented that must be made easy for the pilot to comprehend. Human 
instincts of motion and perception are honed from living in a 1-g environment with the 
majority of motions constrained to the (comparatively) two-dimensional ground. Also, 
lacking eyes in the back of our heads, our natural sense of attention is focused forward. 
While these instinctual characteristics serve well in travel near the ground, they do not 
apply to orbits or to deep-space flight. 


Orbit 


Orbits around the Earth—or any gravitating body, for that matter—present stable, 
constant energy situations. Orbits are convenient parking locations. A vehicle does not 
need to expend energy to stay in orbit indefinitely (unless drag forces from the 
atmosphere or long extensions of the vehicle come into play). Orbits, therefore, are 
common trajectories to select when loitering near gravitating bodies. But so far in the 
course of human evolution, developing an innate sense of placing a vehicle into an orbit 
does not exist. Although a human can instinctively run at just the right speed and 
direction to catch a ball thrown toward them, such natural instincts do not apply to 
placing a vehicle in orbit. Therefore, display systems will be required to provide readily 
interpretable cues for the pilots to transition into orbital flight. This implies presenting 
the natural relations between orbital altitude and orbital speed. This challenge is 
compounded since such cues must naturally blend with the mation cues used when 
flying near the surface. 


Deep-Space and Interstellar Flight 


Deep-space flight adds yet another challenge; namely, the almost total absence of 
familiar cues for motion, position, and orientation. Given the extremely large distances 
between astronomical objects and that relativistic effects do not become significant 
(>1% distortions) until reaching beyond 10% of light speed, the view outside the craft 
will appear stationary—even when traveling at 60 million miles per hour (9% c). The 
display systems that are tied to the navigation references (to be discussed later) must 
convey motion to the pilots in a natural manner despite the absence of familiar human 
cues. 


Compounding the absence of a sense of motion, there is an absence of orientation. 
There is no dominant direction for "up" during deep-space flight. If some form of 
artificial or synthetic gravity is provided for long-duration crew health, then that 
internal 1g will create the most dominant sense of “up" for the crew, and the display 
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system that conveys the spacecraft's orientation relative to the external space will have 
to be clear enough to overcome this prejudicial sense of orientation. 


Notice, for example, that in almost all science fiction stories, spacecraft move /faterafly 
(forward) relative to the vehicles' interna/ sense of “up” (Figure 7}. Motion along the z- 
axis is seldom mentioned. Although this is a natural extension of how we move relative 
to the surface of the Earth, it is not the only scenario. In contrast, consider a rocket 
whose 1-g orientation is aligned with its major axis of motion. This is a consequence of 
its propulsive thrust. In other words, at least two conventions for direction during deep- 
space flight are possible: the notion of lateral motion across a landscape (where the 
internal 1 g is at right angles to flight), or vertical motion with an astronomical range 
(where the internal 1 g is coincident with the direction of flight). 


Conveniently matching film studio conditions, the interiors of fictional spacecraft 
provide a comfortable 1-g environment for the crew. They also follow the terrestrial 
convention of motion: their major direction of motion is forward (a lateraf motion), 
even though they are experiencing an acceleration force of 1 g upward (their internal, 
synthetic gravity). These two directions, up and forward, are at a right angle. In 
contrast, the thrusting direction and the internal g-axis of a rocket are in the same 
direction. The choice of orientation for rea/ deep-space motion is a subject for further 
study. 


Figure 7. Typical Science Fiction Orientations. [Images: A. Szames] 


Since propulsion breakthroughs have not yet been discovered, there is no way of 
knowing if the propulsion methods themselves will dictate the choice of orientation. 
Therefore, to plan for the uncertain future, this is a choice worthy of deeper study. Is 
the natural human instinct for forward-dominated motion a better human-machine 
interaction than the upward-dominated motion that might be dictated by the propulsion 
method? Such an assessment must also consider how well the convention works when 
transitioning from deep-space flight into orbit, then landing, and then back again to 
deep-space flight. Once any convention is set into place, it will be difficult to change 
later. 
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Crew Size Considerations 


The last aspect to take into account as a consequence of mixed operational regimes is 
that of the crew size. For short-duration missions (less than a few hours), it is 
reasonable to conceive of vehicles with only one pilot. For more complex missions, 
additional crew will be required, and thus additional displays and controls specific to 
their tasks will be required. Finally, for long-duration missions, sufficient crew will be 
required to carry out its mission and maintain optimal vehicle performance. These 
changes—for accommodating the roles and responsibilities of crew in relation to the 
overall mission—are likely to be the same as those distinctions in traditional vehicles 
(e.g., cars versus cruise ships). Those changes typically include a hierarchical 
organization, which is independent of the issues of propulsion physics. 


Essential elements will include monitoring and controlling the 1-g internal life-support 
environment as well as ensuring the long-term health of the crew. 


Full Span of Speeds 


In addition to inertial effects previously addressed, the implications due to high speed 
remain. Accommodating the reaction time of the pilot is critical. The extreme high 
speed of breakthrough spacecraft will demand that automated flight controls take 
precedence over the pilot's manual flight control. 


Automated controls for aircraft and even for automobiles are an ever-improving 
technology. For breakthrough flight, these technologies will be mandatory and will also 
have to include options for maneuvering near ground, into orbits, and through deep 
space. This should come as no surprise, since the advantages of having automated 
flight controls warrant their use even if pilot reaction times were not an issue. 


Table 1. Comparing Reaction Time to Distance Traversed at Various Speeds* 


Walking 


Driving Around Town 


Commercial Air Flight 
Hypersonic Flight 4,000 6,400 0.00001 5,900 1,800 1 2 
Low Earth Orbit 17,500 | 28,000 | 0.00003 | 26,000 7,800 | 5s | 8 | 


eee ee 
Nonrelativistic Flight Po ae 89 Million 27 Million 17,000 27,000 


aera 400 650 Hi ‘ig 110 180 
Relativistic Flight Million mea 590 Million | 180 Million 


ne ER 


! The distances traversed while waiting for the pilot to react are reasonable for speeds slower than hypersonic 
flight. If traveling at hypersonic speeds near the ground, however, the situation |s different, At some point, 
regardless of the skill of the pilot, an automated system will be needed, Also note the huge disparity between the 
fastest achieved speeds (deep-space probe) in comparison to nonrelativistic flight. This disparity of three orders of 
magnitude is a clear statement about the state of our technology when contemplating deep-space flight. 
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gat As 5 660 1.07 ates sate 180 297 
Extreme Relativistic | Million Billion 970 Million | 270 Million Thousand.) “Thaweana 
p f 6. & 70 1.08 : oes 190 300 
Eight Speed j Million Billion seu Millen | SOG ilion Thousand Thousand 
F 13 22 ae ies ma 
Faster Than Light? Billion Billion 202 Billion 6 Billion 4 Million 6 mtition | 


Table 1 is designed to put a pilot's reaction time into perspective; it compares distances 
traversed during the one second it takes the pilot to scan and comprehend his displays 
(“dwell time’) and then react.* In addition, it will take time for those commanded 
changes to take effect, but those durations are not known. The distances shown in the 
table are those traversed before any corrective actions are initiated. If these distances 
are determined to be excessive, then automated flight controls are mandatory. 


Another aspect resulting from the effects of ultrahigh vehicle speed is the so-called 
"relativistic twin paradox."!9 Because of relativistic effects, there will be a mismatch 
between the time measured aboard the craft and that measured at its base of 
departure. The equations to track this situation are well established.!1 The challenge is 
how to present this information so that both the crew and the mission personnel at the 
base can easily comprehend the implications. 


More provocative than the implications of relativistic speeds is the possibility of faster- 
than-light (FTL) travel. Beyond the perplexing issues of causal violations and closed 
time-like curves inherent with all FTL notions to date,!* there is the question of tracking 
position, orientation, and motion when beyond light speed. 


It is reasonable to assume that when a vehicle is traveling FTL, the normal flow of 
electromagnetic waves (i.e., light) to and from the craft will be cut off. To better 
visualize this, consider the Doppler shifts as a vehicle approaches light speed. The 
colors of light heading into the flight path will be shifted to such a short wavelength 
that it will cease to be detectable. Similarly, the light approaching the rear of the craft 
will red-shift so much that it also ceases to be detectable. Again, do not count on 
windows. 


Navigation References 


The main difference between navigating with existing vehicles and breakthrough 
vehicles is that the breakthrough vehicles will have to navigate in deep space and 
around other astronomical bodies where GPS systems and location beacons do not exist. 
Another major difference is that the physics of the propulsion methods might distort or 
block information that is traditionally used for navigation. 


inertial Navigation (Acceleration Measurement) 


In inertial guidance systems, accelerometers and ring laser gyros accurately track the 
changes in the vehicle's motion (lateral and rotational accelerations). These signals are 
integrated to keep track of position by evaluating changes in both velocity and 
acceleration. 


In the case of the double hull, where the inertial effects might be different inside of the 
craft, these tools become more difficult to apply. If we assume that the physics and 
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technology for manipulating inertial fields (or for warping space-time) can accurately 
track these effects, then that knowledge may compensate to keep these tools viable. 
Design of future guidance systems must address this issue. 


Absolute Velocity — Universal Speedometer 


Conveniently, nature provides another reference frame for deep-space navigation. The 
cosmic microwave background is a reference frame against which velocity can be 
measured relative to the mean rest frame of the universe. By comparing fore/aft 
Doppler shifts relative to this highly isotropic and homogeneous radiation, velocity 
measurements can be derived. For example, the net velocity of the Earth’s motion 
relative to this background has been measured to be 365 km/s.!7 However, the cosmic 
microwave background will not be detectable at FTL speeds. 


As illustrated in Figure 8, although many of the pictures of the cosmic microwave 
background radiation remove the prominent dipofe moment shown in this graphic (the 
major color difference), it is precisely this dipole—the difference between fore/aft 
Doppler shifts—that provides a navigation reference for deep-space flight. The 
projection of this image is a spherical shell that has been opened and flattened. The 
Doppler shift corresponds to the Earth's motion of over 1.3 million km/h relative to the 
mean rest frame of the universe. The Earth moves in the direction away from the red 
and toward the blue. . 


Figure 8, Cosmic Microwaves as Universal Motion Reference Frame. (Credit: NASA] 
Position-Reference Star Trackers 


For deep-space flight, the star trackers that have been developed for existing 
spacecraft may still prove viable, new instrumentation will probably be required. Given 
the enormous expanse of space, the apparent locations of stars will not vary that 
much.4* Even those that do appear to move—our closest stars—are known well enough 
so that software can take into account how those positions will change as the vehicle's 
position changes. Due to Doppler shifting, as noted previously, checking positions 
relative to the stars will only be possible at sublight speed. 


For speeds approaching light speed, corrections will be required for relativistic effects. 
Such effects will probably not become apparent until traveling well beyond about 9% c, 
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which is a speed that is still three orders of magnitude beyond the highest speeds 
achieved to date. 


Another modification for star trackers will be required for FTL travel. In essence, with 
FTL flight, the vehicle arrives at the destination ahead of time—in an unfamiliar way. To 
understand this, recall that a// information we see from the cosmos is old. Those images 
have taken a while to reach us, and the reality at their point of emission has continued 
forward in time. For example, when we see sunlight, the image is more than 8 minutes 
old. The images we see from Alpha Centauri show what it looked like over 4 years ago. 
Thus, if we could zip to Alpha Centauri instantly, over 4 years of time would have 
elapsed since we last looked at it. Alpha Centauri's condition will be a surprise upon 
arrival. 


Therefore, any star tracker to accompany FTL flight must take into account the 
trajectories of astronomical objects so that their positions can be accurately predicted 
to correspond to the correct time of arrival in both spatial and temporal coordinates. | 
There is no known precedent for this situation. 


In support of the forgoing discussion, we are speculating that heretofore unknown 
advances in physics regarding the quantum vacuum and the nature of inertial frames 
will result in new motion-detection technology. In researching future propulsion 
breakthroughs, the utility of sensing and affecting such phenomena is pertinent. 


Compilations of Implications 


The following list is a compilation of the characteristics discussed in this section about 
the possible features associated with breakthrough flight. While the list is admittedly 
incomplete, it conveys the most significant differences compared to conventional 
methods of flight. 


e Six degrees of independent motion/orientation: 
- Translational motion: fore/aft, left/right, up/down. 
— Rotational (orientation): pitch, yaw, roll. 
e Distinct inner and outer environments for inertial and gravitational forces. 


® Speeds encompassing zero, subrelativistic (<0.1 c), relativistic (Q.1 csv <1.0c), 
and beyond light-speed, yet expecting a travel limit of about a 100-light-year radius 
around the Sun. 


e Three flight regimes: 


~- Near the surface of gravitating body (where gravitational direction provides 
natural orientation). 


— Orbits around a gravitating body (where cues for entering orbit are required for 
the pilot). 


— Deep-space flight (without obvious orientation cues or obvious sense of motion). 
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® Navigational information sources: 
- Position: 
« Master reference taken relative to starting position (Sun-Earth system). 
» Current location taken from the following: 
o Star tracker: 


¢ Modified to handle 3D database of star locations for deep-space 
motion (yet less than 100-light-year radius around the Sun). 


¢ Predictive trajectories (to extrapolate positions for FTL travel). 
¢ Note: Can only take star tracker readings at sublight speed. 


o First integration on measured velocity and relative to point of 
departure, 


6 Second integration on measured accelerations since point of departure. 
— Velocity: 


= Directly measured from cosmic microwave background Doppler shifts, 
where velocity is relative to the mean rest frame of the universe. 


» First integration on measured accelerations since point of departure. 
— Accelerations: 
=» Linear: 


o Accelerometers with corrections calculated based on the influence of 
the propulsion methods that affect gravitational and inertial forces. 


o Differentiation of velocity changes as measured using the cosmic 
microwave background. 


» Rotational: 


o Gyros (e.g., ring laser gyros) with correction inputs from the 
propulsion methods that affect gravitational and inertial forces. 


o Orientation as inferred from star tracker measurements. 
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Chapter 2: Human-Machine Interface Lessons 


Over recent decades, substantial improvements have been made to human-machine 
interfaces.1° Most of this progress relies on better accommodating the norms and limits 
of human perception—lessons that do not change even when vehicle characteristics 
change. These lessons are reviewed in this chapter and then applied in the conceptual 
design offered in the third chapter. Examples of such characteristics include reaction 
times, tunnel vision under stress, instinctual association with position, interpretations of 
displayed colors, and lessons learned from interactions with display and control 
technologies. 


Recent progress on augmented reality displays, ‘© voice control, }”? gesture-based 
computer inputs,48 and brain-machine interfaces (BMIs)!° are also considered. In this 
study of far-future possibilities, these technologies are assumed to have reached full 
maturity. Instead of going into the details of their status, only their implications will be 
addressed here. 


One exception was made when considering emerging technologies—specifically the 
notion of modifying humans for breakthrough flight. This exception includes brain 
implants for BMIs and reengineering humans (transhumanism) to adapt to new 
requirements.*° Rather than requiring humans to be reengineered for breakthrough 
flight, this study focuses on adapting the cockpit to address natural human 
characteristics. This forces attention on the cockpit design requirements. 


Numerous references about human factors were consulted, focusing on those details 
most relevant to this study. Since similar assertions were echoed in many of these 
references, it is not always clear how to trace a given assertion to a specific reference. 
Instead, an annotated bibliography is included at the end of this report that has short 
descriptions of each reference. 


HUMAN PERCEPTION NORMS 


Physical Object Analogs 


Human interpretation mechanisms are rooted in the paradigm of a_ physical 
environment. Mimicking a physical environment in a display and control system 
enhances comprehension and allows features to be recognized with less effort than 
when translating dials, bar graphs, or alphanumeric displays.2! This not only pertains to 
perceptions of motion, but also to the comprehension of a vehicle's operation: 
understanding its limits, supplies of consumables, malfunction modes, and other 
parameters. 


Accordingly, humans naturally remember where to look for a particular piece of 
information or what direction to flip a particular switch. Humans are also adept at 
subconsciously applying models of social behavior and causal events. Hierarchies, 
similar to societal organizations, offer a natural model with which to categorize systems 
and subsystems. Causa/ relations (cause leads effect} match well with procedures and 
operational flow diagrams. For more abstract concepts and complex data sets, a 
combination of hierarchical and causal relations can be used, typically taking the form 
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of conceptual maps. Numerous templates for these tools are readily available and 
many are adaptable to gesture-based interaction.?2 


Color 


Color is convenient for labeling different categories of information, providing status 
indications, and for adding redundant (corroborating) information to minimize errors. 
Humans mentally process color information first and do so more accurately and with 
less effort than when distinguishing shapes or alphanumeric indicators. The upper limit 
on the recommended number of colors to simultaneously display is only about 4 to 12 
colors. To augment this number, it is acceptable to adjust the saturation or brightness 
of a given color to represent gradations, such as terrain height or the criticality of a 
variable.2? 


Although research has not progressed to the point where absolute recommendations 
can be made for the use of colors, the tripartite system is a widely recognized and 
easily discernable status indicator: 


e Red = danger. 
e Yellow = caution. 
e Green = safe to proceed. 


Conveniently, fewer pilots have colorblindness than the general public, and thus color- 
coded displays can be used more effectively than with the general public (data: 8% of 
males and 0.4% of females are colorblind to red-green or biue-yellow distinctions).24 


Other lessons regarding the use of color include the following:25 


e Use strong colors sparingly: strong colors draw attention, and too many of them can 
overstimulate the user. 


e Yellow is a rare color since it is simultaneously soft and intense. 

e The color of objects is more discernable than the color of text or lines. 
e Use soft colors (grays, pastels) for backgrounds and large areas. 

e A good color choice is one where that color is not conspicuous. 


e Use soft contrasts or graduated contrasts between areas, since hard contrasts can 
appear to vibrate. 


e Avoid having the color white next to strong colors since it is too much of a contrast. 


e Avoid framing a box of text since that creates clutter. Instead, place the text in a 
box with a different fill color. 


e If needed to strengthen boarders between softly colored areas, frame the area with 
a darker shade of its fill color. 
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Fields of View 


There is a difference between what humans are physically capable of seeing and where 
they naturally concentrate their attention. Using a reference point that is straight 
ahead and level with the eyes, the normal full field of view extends laterally (left/right) 
+100°, upward about +60°, and downward about -75°. Of course, turning the head 
can extend the lateral reach directly behind, but that rearward view would be in the 
edge of periphery vision. This full span, however, is not where humans normally direct 
their attention. 


With the head in a fixed position, the normal field of view spans left/right 100°. The 
periphery is most sensitive to motion even when discrete images cannot be resolved. 
The region shown that spans roughly +60° is the area that can be easily scanned by 
moving the eyes and whose information can be mentally processed in parallel. A much 
narrower field, about £6°, is where humans can discern details and begin processing 
information serially. This zone can be aimed anywhere in the normal field of view. 
Finally, the region of focus where alphanumerics can be read is only about +2°, which 
corresponds to only one square inch at a distance of 2 feet. Another characteristic of 
human vision is the tendency to fixate in the upper right corner of a given display. In 
the figure insert, the percentages shown refer to the proportion of total time that the 
eye tends to fixate on those regions. 


Normal field of view | Area where details can 
fale eg re ge i be discerned (Macula) 


Area of easy eye 
motion scanning 


Figure 9, Human Fields of View. [Graphic: A. Szames] 
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Peripheral Vision 


Peripheral vision has been used with some success to display rates of change and 
artificial horizons in aircraft. Evidence indicates that peripheral vision can process 
spatial information in parallel without appreciable mental attention. Also, evidence 
indicates that peripheral spatial cues (e.g., artificial horizon laser trace) may still be 
subconsciously processed during stress-induced tunnel vision, even though the pilot is 
no longer consciously noticing it.26 


Attention 


In the absence of stimuli, visual attention is spread evenly across the full field of view, 
and that information is processed subconsciously in parallel. But in the event of motion 
or noise, visual attention aims toward those changes, and then that information is 
processed serially. In the context of cockpits, this means that all the panels and 
windows that are normally in the field of view are processed subconsciously in parallel, 
and a change in any one of those will be noticed, thus drawing attention to that change. 
Once attention has been triggered, the information is processed more serially. 


Blinking lights are a common way to draw attention. Sound can also be used, and the 
combination of sound and lights is commonly used in malfunction enunciator panels. It 
is possible, however, to saturate the pilot with too many blinking lights and sounds. 
Although firm values are not established, it is recommended to keep such functions to a 
minimum—preferably tied to the highest-priority status indications. 


Another method to draw the attention is through physical feeling. Vibrations or a 
change in fee/ of the vehicle will get the attention of the pilot, and with experience, the 
correlation between physical sensations and the status of the vehicle can become 
second nature. Historically, there are many instances where the pilots were innately 
able to sense changes in operating condition of the vehicle just through feel. 


In addition to naturally created sensations, having deliberate vibrations built into the 
seat is another option for sending information to the pilot. Force-feedback controls 
have also been found helpful, where the degree of resistance or vibration fed back 
through a control (e.g., joystick and pedals) provides interpretable information that can 
be mentally processed in parallel. 


Upon the advent of control over gravitational and inertial forces, it will likely become 
possible to deliberately provide the pilot with vestibular cues—mimicking inertial 
accelerations in association with the external conditions, but at survivable levels. 


DESIGN FOR STRESS 


The most important time for the pilot-machine interface to work optimally is in 
moments of crisis. Thus, as a starting point for cockpit design, it is best to focus on the 
highest-priority information and controls and to present those displays and controls in a 
manner that accommodates human norms during stress.2” Accordingly, this section 
covers human limits and errors and advice for providing alarms and response options. 
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Dwell and Reaction Time 


It takes about 0.4 to 0.6 seconds of dwei/ time at a particular display to extract the 
necessary quantitative information and at least 0.125 to 0.2 seconds for a qualitative 
recheck to reaffirm the reading has not significantly changed. In addition, the pilot 
needs another 0.125 to 0.2 seconds to act on that information.2® Taken together, this 
creates a total response time of 0.7 to 1.0 seconds between first looking at a display 
and commanding the appropriate response. 


As an aside, displays whose update rates exceed this dwell time cannot be accurately 
read. This means that displays should be slowed down to the rate at which humans can 
absorb their information, roughly a half-second. 


Tunnel Vision 


While under stress, humans tend to narrow their visual attention, often fixating on a 
single central display or task: the greater the stress, the greater the narrowing. 
Humans are oblivious to this effect as it is happening. (As an aside, hypoxia—breathing 
insufficient levels of oxygen—induces the same effect and can be used to simulate this 
condition during training.) Peripheral vision is ignored, although some evidence 
suggests that spatial peripheral cues are subconsciously retained. This tunnel vision 
tendency cannot be prevented, but can be accommodated by providing the most critical 
information on the central display in a natural symbolic format.29 


Forgetful Visual Scanning 


Humans tend to lose track of when they last looked at a particular area of information, 
sometimes forgetting to recheck parameters when needed.?° This is where automated 
checklists can help,*? as well as vehicle management systems that process the vehicle's 
situation and then recommend the best corrective measures. 


Limiting Options 


Humans tend to be able to retain only about 3 to 10 different items in short-term 
memory. Some studies focus on seven items as the optimum. Thus, it is recommended 
in quick-selection menus, or when designing hierarchical categorizations, to have no 
more than four to seven items per level. For groupings of information that are not 
needed during critical moments, these constraints can be relaxed, but with the added 
consequence of requiring significantly more browsing time. 


Adaptive Display Errors 


In computer displays having multiple layered windows, or where the display changes in 
different situations, it is common that a user will lose track of how their current display 
image relates to the whole system.*4 This problem is called "getting fost" or referred to 
as a "keyhole" error. To prevent this error, it is best to simultaneously display some 
schematic indicator of where the user is in the system. This requires a_ pictorial 
representation of the relations of the system’s operational windows. This situation is 
consistent with the evidence that humans tend to rely on a physical model for where to 
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look for information rather than being proficient at remembering a sequence of steps 
from which to retrieve information. 


In systems having different operating modes, a common error is for an operator to 
execute a command sequence that is inappropriate for the mode they are currently 
using but entirely correct for a different mode.%3 This type of error is called a "mode 
error." An example of this is when a pilot enters a new heading for the autopilot to 
follow when the plane is not in autopilot mode. To prevent this error, it is advised to 
have the most critical operating modes as fixed, physical displays. 


Other Distractions 


It is important to remember that the cockpit might not always offer a smooth, 
distraction-free ride. Buffeting can cause a finger to press the wrong button (or a 
gesture-based command to misdirect), or the eyes might not be able to resolve a 
particular value. Excessive use of audible alarms and blinking lights can saturate the 
pilot. Other activities or distractions available to the pilot must be taken into account so 
that the most critical functions are easy to find and operate, including sufficiently large 
buttons and text. 


Alarms and Responses 


Traditionally, physical enunciator panels combined with audible alarms and blinking 
lights were used to highlight malfunctions. In addition to fixed enunciator panels, more 
complex systems can now computationally analyze a number of variables and only 
present the most pertinent values and alarm states. 


DEVICES TO CONVEY INFORMATION 


Observation and interpretation of displays is always secondary to the primary task of 
actually operating the system. Thus, it is important that displays be designed to first 
serve the user and to take on as much of the information-processing burden as 
possible. Also, it is suggested that the displays and control input devices should be 
designed according to the operator's preferences, as opposed to the more common 
practice of designing based on the system engineer's expectations. The following 
paragraphs describe lessons learned regarding various methods of conveying 
information to the pilot.24 


Fixed and Adaptive Displays 


Fixed displays, such as the gauges of older aircraft, do not change the type of 
information they display nor do they morph their format. These can be dials, bar 
graphs, an artificial horizon, or simple indicator lights. Such fixed displays can be 
projected with computerized screens where several interrelated values can be combined 
into a single, computed value. Adaptive displays, on the other hand, can change their 
formats or change the topics displayed to fit operational situations or individuals. Both 
have their respective optimum uses. 


During the advent of adaptive displays (circa 1990s), they tended to be overused. The 
advantage is that the information can be tailored to minimize the volume of information 
displayed at any one time. The disadvantages are that such systems require more 
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effort on the part of the operator and are more prone to mode errors and keyhole 
effects, particularly during high—stress situations. It was also found that automatically 
changing display formats added anxiety and mistrust of the system, because the user 
was now burdened with the additional task of determining why the display just 
changed. A survey of Boeing 757 and 767 pilots in 1989 revealed that over half felt 
their workload had actually increased by the high levels of automation introduced into 
these cockpits. In practice, users forced adaptive displays into fixed displays to improve 
their utility.7° 


The ultimate lesson is that the most critical displays should be fixed to take advantage 
of the naturally efficient search method where humans use mental models that are 
analogous to physical objects. For other functions that are less time-critical and where 
further details are required (e.g., entering destinations and way points into a flight 
plan), adaptive displays are more advantageous because they require less console 
space. Other lessons include the following:7° 


* Following the same recommendations for the individual elements of adaptive 
displays that are given for graphical and alphanumeric display components. 


e Matching the performance characteristics of the displays to the operator's own 
internal model of the system, and they should be structured to accommodate a 
user's knowledge. 


® Packaging information into appropriate, meaningful clusters. 
e Representing logical and causal relationships should be implemented. 


*e Using “fewer displayed variables with many states” is recommended over the display 
of “many variables with few states.” This necessitates arranging variables ina 
hierarchy where the lower-level variables are combined into higher-level variables. 


e Using color to improve the speed of human comprehension. 


® Accessing back-up information should be easy (familiar procedure), and it should be 
easy to browse through the system without disturbing its operation. 


Virtual Displays and Augmented Reality 


An extension of the adaptive screen is a virtual display where the information is 
projected into a field of view without the need for a physical display screen. These can 
function as adaptive displays or be used with gesture-based input systems. 


Although far-future cockpit technology is still being researched, this study assumes it to 
be fully matured, such that any information can be displayed clearly in the pilot’s full 
field of view. That being said, the other recommendations about accommodating 
human norms and the lessons of adaptive displays should be heeded. By that, it is 
meant that the critical information should remain consistent, and the functions that can 
tolerate slower user actions can be accommodated with pop-up screens. Also, rather 
than relying solely on a virtual display, the critical displays should still have a physical, 
redundant set of displays. 
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A key feature of virtual displays will be the inclusion of augmented reality. This means 
that key elements in the field of view are highlighted with color or framing to draw 
attention?” to them. This includes adding obvious border lines to objects for which the 
pilot needs to be aware. Similarly, a virtual display of the environment outside the craft 
could be augmented to reveal characteristics outside the range of normal human vision 
such as infrared and ultraviolet light, or objects too small or fast to normally be noticed. 


Graphical and Alphanumeric Representations 


Pictorial displays in combination with alphanumeric displays have been found to be an 
effective way to convey information to a pilot. A pictorial representation of the item or 
condition being controlled (maps, artificial horizons, graphical representations of vehicle 
stores, and others) takes advantage of the interpretation paradigm of physical models, 
and thus enhances comprehension and allows features to be recognized with less effort. 
Although reading alphanumeric displays are more time consuming and workload 
intensive, they provide more accurate values and are more easily relayed verbally to 
secondary users (i.e., the value can be read aloud). It has also been found effective to 
place the alphanumeric values in the upper-right corner of the graphical display. Other 
lessons include the following:7® 


e Using consistent text, format, placements, axes, and other parameters.?? 
Consistency has been found to be more desirable than optimized displays.*° 


* Displaying parameters relative to their expected values rather than just displaying 
the alphanumeric value. Some indication of the criticality of an off-nominal reading 
also needs to be displayed. 


e Using zero-value points as a reference and showing range settings on graphical 
displays to indicate how a given value compares to the desired or predicted value; 
all ranges should be roughly the same size if possible. 


e Displaying nonessential symbols as half-intensity helps declutter displays. 
Enunciators 


Historically, all the major alarm indicators for a system were grouped together into 
what is called the "“enunciator panel.” Usually, this panel is an array of rectangular 
lights with each representing a possible alarm or critical-state variable (e.g., 5 x 5 
arrays of 25 discrete alarm channels). Frequently, the lights use the tripartite color 
convention, blink to alert, and are connected to an audible alarm. There is usually some 
means to acknowledge the alarm and thus stop the distracting blinking and noise 
without canceling the alert status indication (e.g., goes from blinking with audible alarm 
to just lit in red or yellow). 


Old hardware displays unintentionally maintained a manageable number of alarm states 
as an inadvertent, but positive, consequence of the difficulty of adding enunciator 
segments. With computerized systems, however, this number can grow to be 
overwhelming. Conversely, with computerized systems, analysis of vehicle operations 
can reduce a large number of variables into a composite key status variable for alarm 
states and offer pre-determined options of emergency responses to the pilot. Hence, 
the concept of an enunciator can be quite useful if following these recommendations: 
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Limit the number of alarms. 
Display alarms in a dedicated, predictable location. 


Link alarms to a pictorial representation of the system to help recognize root causes, 
causal relations, and where to focus corrective action. . 


Provide a means to acknowledge the attention-getting aspect of the alarm without 
turning off its alert status indication. 


In the case of multiple triggered alarms, the system's most prominent display 
should be the highest hierarchical alarm, the highest causal alarm, or the location 
on which to focus corrective action. 


Link the alarm clearly to the control response options. 


Checklists 


Checklists are a standard and useful tool in the operation of vehicle systems. They can 


be 


displayed on adaptive displays where the list corresponds to the vehicle's current 


status (e.g., preflight readiness or postflight checks). Recommended features of a 
checklist should include the following:*! 
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Display the desired and current value with each checklist item. 


Require that buttons/switches have to be touched to acknowledge that each item 
has been checked off. This helps avoid skipping items and helps the user keep a 
sense of involvement in the process. 


Include a completion call at the end of the checklist. 
For long lists, subdivide using the following guides: 
— List critical items first. 


— Use geographical (i.e., physical location) flow or pictorial flow to help users 
retain relation between the model of the system and the procedures being 
performed in the checklist. 


— Use parallel tasks between the onboard operation and the ground operations 
to maintain fluency between pilot and mission control communications. 


— Include buffers in checklists to provide recovery time for anomalies. Decouple 
tasks if possible so that the failure to meet a given checklist item does not 
leave a parallel item in an open state. 


— Standardize checklists within the system (assuming nested, hierarchical, or 
multiple checklists) to minimize the mental burden necessary to extract 
information when going from one format to another. 
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DEVICES FOR RECEIVING PILOT COMMANDS 


In much the same way that displays are configured to match the dominant norms of 
human behavior, so too are the command functions. As evidenced by the gaming 
industry, the motions of the pilot's hands—whether by a joystick or a Wii controller4?— 
mimic the intended physical motions of the object under control. Some of these control 
technologies are described below. 


Physical Controls 


Devices such as joysticks, toggle switches, thumb wheels, rotary switches, and even 
keyboards will still be mandatory in cockpits of the future. This is based on needing 
fixed locations for the most critical displays and controls. Additionally, tactile feedback 
helps the pilot know that their command has been entered. In moments of crisis, a 
human can react quickly to reach for just the right switch and detect the sensation 
when that switch is flipped. 


Joysticks take advantage of human nature, where hand motions mimic the intended 
motions of the object under control. Joysticks and pedals with force-feedback or 
vibration feedback add another element of information that humans can process in 
parallel—feedback that would not be possible with virtual controls. 


Despite advances in other data-input technologies (e.g., voice), it is likely that there 
will be times when a keyboard is required, but its routine use is not expected. 
Keyboards are an efficient way to accurately enter alphanumeric data and especially 
narrative text. Conversely, keyboard use is time consuming, physically requires a large 
space (can be stowed, however), and is subject to errors during vibration or buffeting. 
Such errors are reduced when having some physical support to help anchor the hands. 


Gesture-Based Inputs 


By the time that propulsion breakthroughs become viable, it is likely that gesture-based 
commands will have evolved past the current systems’ problems of misinterpreting 
wayward motions and will have become an effective way to replace the mouse for 
cursor control. In addition, it is expected that more options will be available through 
gestures than through existing mouse buttons (right click, left click, and scrolling). 
When used in combination with a voice-command system, it is expected to be an 
effective tool for the more complex and varied instruction sets, such as navigation. For 
example, the notion of being able to point to a location on an expansive virtual map 
and say, "go there," seems an ideal implementation. 


Furthermore, the use of gesture-based inputs in analyzing new data seems appropriate, 
provided that the lessons frorn adaptive displays are heeded. Although fascinating, 
gesture-based commands are dependent on how well the information that they are 
manipulating is organized. Therefore, these inputs might be prone to the same keyhole 
errors and mode errors of adaptive displays. 


Prior lessons regarding quick emergency commands should also be heeded, specifically 
where all critical commands have dedicated physical controls. Gesture-based 
commands can be redundant, but again, the physical control should be the dominant 
source of critical inputs. Consider the event of buffeting, where the position of an 
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extended arm is likely to waver. This would induce significant errors in the gesture 
system. : 


Voice Commands 


Voice commands are promising for hands-free and parallel task initiation.*5 By the time 
that propulsion breakthroughs become viable, it is likely that voice-command 
technology will have solved the problem of voice recognition (including when stressed) 
and the problem of accommodating the myriad of ways to ask for the same thing. That 
having been considered, voice commands are projected to be an_ excellent 
augmentation to the set of other cockpit controls, particularly with the more complex 
and varied instruction sets, such as navigating within gesture-based systems. 

Based on lessons from human-to-human communication, however, it is advised to have 
the pilot’s commands repeated back to them as a form of confirmation. | 


Brain-Machine Interface 


Although research is currently in the initial stages,*® this study of far-future cockpits 
assumes that this technology is fully matured, such that the pilots can issue commands 
with their thoughts. A limiting constraint here is that only non-invasive techniques are 
used (avoiding the need to modify the pilots). 


Regardless of the sophistication of the technology, a weak link in these systems is the 
human mind itself; that is, thoughts can wander. Much more study is required to 
determine the implications of distractions, loss of concentration, fright, levels of mental 
discipline, and many others. 


CONTEMPORARY AIRCRAFT COCKPITS 


For comparative reference, some features of contemporary aircraft cockpits are 
examined next. Key displays and controls are becoming standardized, although the 
placement still varies across manufacturers and models.4” Regardless of differences, 
the functions are representative of similar functions to consider for breakthrough-era 
cockpits. 


The layout of the Airbus A380 Flight Deck, shown in Figure 10, has major flight 
functions located in specific areas. There is a combination of fixed and adaptable 
displays. The following items of interest are identified: 


e Primary flight display. 


Navigation display. 


Flight mode control. 


e Vehicle status indicators. 


Joystick, throttles, and keypads. 
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Figure 10. Flight Deck of Contemporary Aircraft. As one example, the Airbus A380 Flight deck contains 
displays and controls that are indicative of displays and controls that will be needed in breakthrough-era craft. 
Image source: <http://www. flickr.com/photos/83823904 @N00/64156219/> (September 2007) 


Primary Flight Display 


The primary flight display (PFD) provides information about the vehicle's orientation 
(pitch, roll, and compass heading), motion (speed and rate of climb), and altitude. It is 
centrally located in front of the co-pilot’s seat, and a smaller version is available to the 
right of the pilot’s navigation display. Figure 11 shows a close-up of a PFD that also 
shows its heritage from the mechanical instruments of older aircraft that have now 
been merged into a single computer-controlled instrument. 


This instrument is vital to the safe operation of an airplane. It displays orientation 
(pitch, roll, compass heading), motion (forward airspeed, rate of climb), and position 
relative to a critical axis (altitude). This particular graphic was selected because it also 
shows the lineage to the individual mechanical displays of prior aircraft; artificial 
horizon, airspeed, compass heading, altimeter, and rate of climb. Per many of the 
recommendations from human factors research, a blend of graphical and alphanumeric 
elements are now used, and the central orientation graphic, the “artificial horizon,” is a 
direct physical analogy to the physical world. 
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Figure 11. Contemporary Primary Flight Display. 
Source: <http://www.freshgasflow,com/physics/aviation_and_anesthesia/aviation_comparison.html> 


An analogous display will be required for the breakthrough-era craft, but with 
modifications to accommodate the full six degrees of freedom and the additional flight 
regimes of orbit and deep space. For example, the artificial horizon and altimeter have 
meaning near the surface of a gravitating body, but it is not clear how effective such 
references will be in flight regimes of orbit or in deep-space flight. 


These orientation cues as well as compass headings are Earth dependent. Far from 
Earth, the value of maintaining these cues is unknown, and substitute cues have not 
been identified. Other orientation standards will have to be developed for 
breakthrough-era craft that will still be consistent with this aircraft convention when 
operating near the surface of a gravitating body. — 


Navigation Display 


The next most significant display is for navigation. Near Earth, this takes the form of a 
map that can be augmented to show terrain and weather. Due to its importance, it has 
a prominent place on the flight deck. Numerous related controls and displays are 
available for allowing the entry of destinations and waypoints into the system to 
augment this one display screen. 


Breakthrough-era craft will require additional navigation cues; these may include those 
that will direct pilots into orbits and those for navigating in deep space. 
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Flight-Mode Control 


In contemporary aircraft, the flight-mode control panel is often located prominently in 
the glare shield. It contains displays and controls for the autopilot and for setting 
headings, speeds, and other operational parameters. In breakthrough-era craft, there 
will likely be no need for a glare shield, since all images of the exterior will be projected 
indirectly; thus, the intensity of the light can be limited. Nonetheless, having a 
prominent location for the flight-mode display is critical. 


Vehicle Status Indicators 


The status of critical operating parameters of the vehicle, such as its fuel level and 
engine temperature, is essential. Note the prominence of those displays in the center of 
the flight deck. However, other than knowing that similar displays and controls will be 
required for breakthrough-era cockpits, it is too early to speculate on their details. 


Joystick, Throttles, and Keypads 


Some sort of physical controls are required to fly the aircraft. On contemporary aircraft, 
these include the joystick (side-mounted), rudder pedals, and throttle controls. Fine- 
tuning controls that manage features such as trim settings and flight heading can be 
located on separate panels. The details of these controls are dependent an the methods 
of flight. For a breakthrough craft, it is reasonable to speculate that six-degree-of- 
freedom controls will be accommodated by whatever method is most natural for pilots. 


Figure 12. Space Cockpit Visions Circa 1959. Taken directly from Seifert's 1959 book, Space Technology, this 
is an artist’s rendering of a "space travel control station.” It is offered here to convey the provisional nature of far- 
future speculations, including the speculations offered in this report. 
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Chapter 3: Provisional Cockpit for Breakthrough Flight 


Applying the strategy of using the most demanding situations to guide designs, the 
provisional cockpit presented here assumes a single pilot. Without any crew amongst 
whom to disburse the workload, such a cockpit must be more efficient and effective. 
The “design for stress” approach is also applied, which drives the design to have actual 
physical displays and controls that are concentrated in a “tunnel vision” zone. The 
elements on these physical panels are predictably fixed, owing to lessons of human 
factors. 


Newer technology such as virtual reality displays with augmented reality and gesture- 
based inputs are also included, along with voice and thought commands. Based on 
lessons from adaptive display errors, and considering the limits of human 
concentration, limits are imposed on the use of voice and thought commands. 


The overall configuration is shown in Figure 13 and consists of a modest set of physical 
panels surrounding a seated pilot, along with a large virtual display surrounding the 
entire area. The most critical displays and controls are condensed into the forwardmost 
panels. 


FLIGHT MODES 


The most fundamental flight mode is the full manual mode. Each subsequent flight 
mode offers increasing levels of automated assistance, up to and including a fully 
autonomous control mode that can operate even when the pilot is incapacitated. A 
provisional set of flight modes (whose buttons and indicators are presented across the 
central flight mode panel from left to right) include the following: 


e Full manual. 

e Manual w/safe assist. 

* Interactive assist. 

e Fully autonomous (keyed to emergency responses). 
Full Manual 


As the name suggests, this is where the pilot has full control over the motion of the 
vehicle through joysticks, switches, gestures, voice, and thought. This is the most 
difficult flight mode for the pilot, and accordingly provides the best situation from which 
to design the quidance displays and to set the requirements for the various control 
inputs. 


In subsequent studies,** this would be the flight mode to specify when addressing the 
challenges of guiding the pilot for entry into orbits and navigating in difficult situations. 


32 7 
UNCLASSIFIED / /POROPFSTCHE esr ones 


UNCLASSIFIED / / POR OPP! esr orr= 


Figure 13. Provisional Breakthrough-Era Cockpit. The spherical zone around the cockpit center represents the 
virtual surround display onto which images of the environment outside the vehicle are displayed. Keeping with 
lessons of prior human interfaces, a minimal set of physical displays are also provided, with the most important 
ones located deliberately in the “tunnel vision” zone in front of the pilot. [Graphic: A. Szames with background 
image through M. Cherfi] 


Manual With Safe Assist 


To prevent the pilot from inadvertently flying into a hazardous situation, this mode 
augments the manual flight controls by taking over to prevent collisions or to prevent 
entry into hazardous zones (€.g., dangerous radiation or gravitational fields too strong 
fo escape). When the system must take over, it would be accompanied by force- 
feedback into the joysticks and obvious indications on the display. 
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Interactive Assist 


This is a further automated augmentation of the prior flight mode. In this case, 
following commands from the pilot (voice, gesture, or thought), the pilot can request 
for the flight assistant to fly the vehicle as desired. In this mode, it is most likely that 
the pilot will not be touching the joysticks, but rather issuing “drive to" commands to 
the automated system. 


Fully Autonomous 


In the event that the pilot is incapacitated or performing non-flight tasks, the fully 
autonomous mode flies the vehicle in whatever flight command mode was last entered. 
This means that the system follows through with previously entered commands, 
including a selection of preset emergency responses, such as “returning to base” or 
“flying to the nearest medical facility.” These emergency response functions may have 
a dedicated physical panel. 


PHYSICAL DISPLAYS 


In keeping with human-factors lessons, particularly the use of physical analogies of 
human perception, the critical displays and controls will be anchored in physical 
hardware in fixed locations. The primary reason for providing an actual physical set of 
displays and controls is to minimize errors during stressful situations. This includes 
tunnel vision, mode errors, and keyhole errors. The controls also include tactile 
feedback. 


Following the precedents of modern flight decks and human-factors lessons, each of the 
panels will have a dominant function as identified in Figure 14. Although the look and 
feel of these panels has not yet been conjectured, their likely functions and features 
can be estimated. 


Sau Vector Motion Flight Assist 
Navigation Display rateraeive 
Display Display 


Survival 
Enunciator 


Emergency 


———_________» Flight Mode Response 


Cabin comm. 
Control Vehicle Status Control 


External “ Internal 
Sensing ‘ ei Conditions 


Dual Six-Degree Joysticks 


Figure 14. Functional Designation of Physical Cockpit Panels. Basic layout of the physical displays and 
controls. The function of each is described in the text. [Credit: M. Millis] 
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Vector Motion Panel (Primary Flight Display) 


The most critical, survival-dependant displays are in the “tunnel vision” zone, directly in 
front of the pilot. The main panel at the center of the display is analogous to the 
primary flight display (PFD) of contemporary aircraft (shown previously in Figure 11). 
In this case, however, it must be modified to accommodate the full six degrees of 
freedom and the additional flight regimes of orbit and deep space. Because of these 
modifications, and to distinguish from conventional aircraft displays, this text 
provisionally uses the term “vector motion" display. 


The main distinction between an aircraft PFD and this vector motion display is that the 
aircraft PFD is biased toward flight near the surface of a gravitating body and for which 
navigation conventions are firmly established (e.g., "up" and magnetic compass 
heading). The vector motion display, which must accommodate orbits and deep-space 
flight will also feature conventional functional subsets for near-Earth navigation. 


Since we have no established standards for deep-space navigation that are applicable 
beyond our solar system. Devising and designing such a system and the appropriate 
display requires further work. 


Orientation standards will have to be developed for breakthrough-era craft that are still 
consistent with this Earth-surface convention. The main axes adopted for this display 
would also be superimposed onto the virtual surround display. This display will include 
information to ensure a safe trajectory, avoid collisions, and handle other situations. 


In addition, visual cues for assisting a pilot to safely enter a correct orbit are required. 
Although it is expected that such maneuvers would be handled by an automated 
system, the more demanding condition, in which the pilot will have to manually enter 
orbit, presents an appropriate design goal for the display and is a subject for future 
work. 


Navigation Display Panel 


The navigation display is responsible for showing the pilot his location relative to base 
and all other points of interest. Following the common practice of aviation, this display 
is located immediately to the left of the central motion display. 


The navigation display will need a new coordinate system to address deep-space flight, 
transition into orbits, and maneuvering near to and landing on gravitating bodies. Such 
an orientation and guidance system does not yet exist. 


It is expected that most gravitating bodies can be conventionally mapped using 
systems analogous to those used for the Earth, where the rotation axis is a defining 
characteristic. The caveat is that not all gravitating bodies are spherical (asteroids and 
small moons). 


Flight-Assist Panel 


The flight-assist panel is located to the right of the vector motion display. Because of 
the variety of functions this panel must perform, it must be in the form of an adaptive 
display. This panel provides information and accepts pilot commands to assist in 
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operating the vehicle and carrying out the mission. It is functionally similar to the 
"flight management system” *9 of commercial aircraft. The pilot needs critical 
information in real time that can be easily interpreted. If anything goes wrong, the pilot 
needs to know the options and probable consequences of each option. A simplified 
depiction of the nature of the emergency is also critical. Predicted consequences and 
risks of each response option should also be succinctly displayed as part of the adaptive 
display options. The pilot also needs to know, unambiguously and quickly, how to select 
any option. The envisioned functions include the following: 


e Automated assessment and display of the overall vehicle operational status. 


e Automated assessment and display of pilot's health and functional status. (If the 
system detects that the pilot is incapacitated, it automatically engages the "get 
medical assistance" emergency response mode.) 


» Provide short-list of anomaly/emergency response options 


e In emergency, time-critical situations, the system will automatically choose its top 
recommended response if the pilot does not enter in a command within the required 
time. 


e Checklists for various procedures (e.g., pre- and post-flight or diagnostics). 
e Monitor progress relative to mission goals and timeline. 
e Provide access to other databases (é€.g., something akin to an Internet search). 


e Provide the option to project a larger version of the display on the virtual surround, 
whereupon gesture-based commands can be used. 


When designing this panel, the lessons discussed in Chapter 2 regarding adaptive 
displays and graphical and alphanumeric representations should be followed. This 
includes hierarchical organization, limit of displayed choices of four to seven, 
simultaneous display of the current mode of the display relative to a model of the 
system, and sound and verbal accompaniment for displayed information. 


Additionally, the functions of the survival enunciator, flight mode panel, and emergency 
response panel are all tied into the flight assistant, but these fixed panels provide only 
the mast critical alarm and response options. One of the emergency response panel 
buttons allows for selection of whatever response option is currently displayed on the 
flight assistant panel, while the other emergency response buttons are fixed functions. 


Survival Enunciator Panel 


On the left side of the flight mode panel, between the center and left main panels, is 
the survival enunciator. For this limited-size enunciator, only those alarms that are of a 
life-threatening nature are listed. Less critical alarms are either handled by the flight 
assistant or the side panels. Audio cues are also recommended for this enunciator, with 
the added feature that the direction of the source of the sound corresponds to the 
location of the alarm. 
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Copying the general organization of the other major panels, this tall and narrow 
enunciator is divided into three sections: 


e f€xternal Alarms (Upper Left): Alarm indicators corresponding to the most life- 
threatening conditions (limit of 4 to 10) that could occur external to the vehicle, 
such as impending collisions, hazardous radiation, or other critical situations. 


° Vehicle Alarms (Upper Right): Alarm indicators corresponding to the most life- 
threatening conditions (limit of 4 to 10} that could occur internally from the vehicle 
itself, such as catastrophic equipment malfunctions. 


e Cabin Alarms (Bottom): Alarm indicators corresponding to the most life- 
threatening conditions of the cabin’s life support (limit of 4 to 10), such as loss of air 
pressure, excessive internal g's, temperature extremes, or pilot health conditions. 


Emergency Response Panel 


The narrow vertical panel between the center and right panel is the emergency 
response panel: a fixed series of buttons for preprogrammed emergency responses, 
including at least one variable button tied to the recommendations on the flight assist 
display. These are deliberately set into the tunnel vision field of view. Each of these 
responses is programmed to perform without needing additional inputs from the pilot. A 
provisional set of these preprogrammed emergency response buttons is as follows: 


e Evasive collision avoidance. 

e Escape to safe loiter position. 
» Get médical assistance. 

® Effect automated repair. 

» Return to base. 

e Flight-assist recommendation. 


One of the emergency response panel buttons is to select whatever response option is 
currently displayed on the assistant panel, while the other emergency response buttons 
are fixed functions. 


Flight-Mode Panel 


Because selecting any of these emergency responses will change the flight mode into 
"fully autonomous" mode, the flight-mode panel is between the emergency response 
panel and the survival enunciator. Because the flight mode is a critical piece of 
information, it is also located between the vector motion display and the vehicle status 
panel. 


In aircraft, the flight-mode controls are often on the glare shield, but in the 
breakthrough-era craft, there will not be a need for a glare shield. The light in the 
cabin is only from the display systems whose brightness can be set to prevent glare on 
the consoles, 
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Vehicle Status Panel 


Below the vector motion panel is the vehicle status panel that contains the most critical 
vehicle status and controls. This is analogous to where fuel levels and engine 
temperatures would be displayed. In this central panel, only the most critical and 
highest hierarchical status conditions would be displayed in a fixed format. 

More detailed, lower-level statuses and controls are on the right side panel and 
available through the flight-assist system. The proximity of this panel to the survival 
enunciator, the flight-mode panel, and the emergency response panel is deliberate. 
Also flanking either side of this vehicle status panel are the cabin controls (on the left) 
and the communication controls (on the right). 


Cabin Control Panel 


The cabin controls are not just for temperature and pressure of the pilot's cabin, but 
also for the inertial and gravitational environment inside the cabin. This panel is 
deliberately located just below the section of the survival enunciator that displays alarm 
states for the cabin, and just adjacent to the vehicle status panel, with the expectation 
that related vehicle functions are displayed near the cabin controls. 


The novel function of including controls for the interna/ inertial and gravitational forces 
is in Keeping with the recommendation of the double-hull configuration from Chapter 1. 
If the pilot wishes to feel a portion of the vehicle's acceleration, that can be dialed in. If 
the pilot wishes to select a nominal background gravitational field of 0 g or 1 g (or 
other options), that can be done, too. 


Communication Panel 


The communication panel is the primary selection point for communicating with base. 
Accordingly, it is located just below the flight-assist system and the emergency 
response panel. An alphanumeric key pad serves both the communication panel and the 
flight-assist system. 


Available settings would likely include selection of communication methods, channels, 
and related functions. Further details are dependent upon factors that cannot yet be 
predicted. 


Pullout Keyboard 


For more intensive alphanumeric data entry, and when time allows, a pullout computer 
keyboard is available. 


External Sensing Panel 


One of the two side panels flanking the pilot seat is the external sensing panel. It 
includes settings for external sensors, navigation cues, and related functions. It also 
includes controls for how the information obtained from the external sensors is 
processed, displayed, and recorded. 
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Both side panels are deliberately not in the tunnel vision zone, since they provide 
additional functions that are less time critical. Each side panel is a mix of fixed and 
adaptive displays. The convention is that the panel on the left (consistent with the 
navigation panel on the left) is for displays and controls related to the environment 
outside the craft, while the panel on the right is associated with the function of the craft 
itself. 


Only some of the sensors and functions are known at this time, and it is expected that 
further advances in sensor technology might provide other abilities that are not yet 
foreseeable. Following the requirements from Chapter 1, this panel will at least have 
controls for the following: 


e External visible spectrum cameras that provide input to the virtual surround display. 
e Augmented spectrum cameras (e.g., infrared or ultraviolet). 

e Star tracker. 

e Cosmic microwave background sensors. 

e Inertial sensors and their related integrations for velocity and position. 

e Object detection and trajectory determination. 

Internal Conditions Panel 


The right panel flanking the pilot seat has more detailed displays and controls regarding 
the vehicle's functions, such as energy, propulsion, and computation. Similar to the left 
side panel, it is a mix of fixed and adaptive displays. 


This panel provides deeper levels of detail for the elements displayed on the vehicle 
status panel, along with control settings for the same. The functions of this panel are 
closely integrated with the flight-assist system, especially in regard to recommending 
corrective actions to vehicle malfunctions. 


Absent the technology for the vehicle's breakthrough propulsion and power systems, 
further details of this panel cannot be suggested at this time. 


Periphery Motion Displays 


The last two physical panels are the periphery motion displays, which are located on 
either side of the cockpit in the pilot’s periphery vision. They are a physical back-up to 
the virtual surround display, where each offers more intense moving grid lines. 


The panels simply display a grid whose motion is synced with the motion of the craft 
relative to the surrounding space. In this way, the pilot gets subtle cues to motions 
that can be mentally processed in parallel with the other displays. Such an additional 
motion display is expected to become necessary because of the added degrees of 
freedom available for the vehicle's motion. This can be particularly useful during hover, 
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where careful changes in position are required that are difficult to infer fram just 
looking at external landmarks. 


VIRTUAL SURROUND DISPLAY 


Responding to the likelihood that direct visual contact to the outside environment will 
not be available, images of the outside will have to be relayed to the pilot via exterior 
cameras and interior displays. Taking advantage of foreseeable advances, such images 
are likely to be in the form of virtua/ displays. This also allows the most critical motion 
and orientation cues, along with augmented reality functions, to be superimposed on 
these images. At this point, the method of virtual display is not important (e.g., helmet 
mounted, projected on screen, or contact lens). The issues addressed here discuss what 
is required rather than the details of how those functions are provided. 


Exterior Visuals 


Since direct visual contact to the outside environment will not be available, images of 
the outside will have to be relayed to the pilot. A system of cameras that can see the 
external environment are required, plus whatever processing is required to seamlessly 
merge these images for projection on the virtual surround display. 


Additionally, it would be advantageous to be able to focus in on and magnify the view of 
any particular area of the display. 


Points of Interest 


Taking advantage of the technology for augmented reality,°° where objects of interest 
can be highlighted on a display, this function will be included in the virtual surround 
display to flag such things as points of interest, objects, and the trajectories of objects. 


Overlays 


Ideally, it would be advantageous to be able to superimpose false-color images 
representing light spectra beyond normal human perception, such as infrared or 
ultraviolet light. 


Data Call Up 


With such a large projection area available, any of the adaptive displays from the 
physical panels can also be projected onto the virtual surround display and then 
manipulated with gesture-based commands. This feature can employ ever-evolving 
techniques of visual information display.>? 


Sound Augmentation 


Audio cues shall be included so that the location of objects of interest and alarm states 
can be spatially inferred, and redundant information to their locations can be projected 
by visual displays. In addition, the vehicle's flight-assist system can speak information 
to the pilot. Techniques of using different sounds, as well as the manipulation of a given 
sound {i.e., changing the repetition rate of a given beeping tone), will be used. 
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CONTROLS 


Similar to the blend of physical and virtual displays, the control methods will also be 
primarily physical controls (joystick), with augmentations possible through gesture, 
voice, and thought commands. 


Vector Motion: Six-Degree Joysticks 


Adapting to human norms of physical analogies and taking advantage of force-feedback 
features, the primary control for the vehicle's motion will be a pair of six-degree-of- 
freedom joysticks, one for both the left and right hands, located at the edge of the arm 
rests. By moving either hand to mimic the orientation or direction of the desired 
motion, the craft will respond accordingly. 


Studies suggest that force-proportional, instead of displacement-proportional, operation 
of joysticks is optimum. Accordingly, the joystick does not require much space to 
encompass its motions. The intensity of the force input corresponds with the intensity 
of the resulting motion of the craft. 


There is, however, a limit to the span of forces that can be input from the hand 
compared to the span of intensity of the vehicle's propulsion. For example, consider 
that the hand can resolve speed settings of +2 mph, but only over a span of zero to 45 
mph, while the vehicle is capable of speeds fram zero to relativistic. Obviously, some 
additional input, analogous to a throttle, is necessary. The challenge for the 
breakthrough craft is that such a throttle is an option for all three linear axes, not just 
the primary axis of motion. With such uncertainty, and not having any simulations run 
on test subjects, it is uncertain how best to provide the directional intensity control. 
Provisionally, it is recommended to have a trigger and/or a combination of buttons that 
the user selects to convey the maximum intensity of the propulsion system that 
corresponds to the maximum force input to the joystick. For example, with a light 
trigger input, the full joystick force might only correspond to a speed comparable to 
driving a car. With a heaver trigger input or buttons selected to provide deep-space 
flight speeds, the maximum force on the joystick might correspond to 0.9 c. 
Accommodating such uncertainties would be a subject of future study. 


Owing to lessons of natural relaxed positions of wrist rotation,>? the neutral palm (with 
the thumb-side facing inward and forward) is at 35° upward relative to the horizontal 
and 25° forward. This deviates from the normally upward-pointed joystick, but recall 
that contemporary flight joysticks are just for rotational axes’ command inputs. 


Gesture Commands 


Gesture-augmented commands are used in conjunction with the virtual surround 
display to identify and act on objects in that display. For example, the pilot can point to 
an object of interest, which is then highlighted, and issue the voice command, "go 
there." This would be an option in the interactive assist flight mode. 


Other options for the gesture-based commands are difficult to define at this time, due 
to the uncertainty of actually operating such a vehicle. Regardless, however, provisions 
toe account for buffeting and other possible input errars would need to be included in 
future plans. 
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Voice Commands 


Voice commands are projected to be an excellent augmentation to the set of other 
cockpit controls, particularly with the more complex and varied instruction sets, such as 
navigating within gesture-based information interfaces. 

Based on lessons from human-to-human voice communication, it is advisable to have 
commands from the pilot repeated back by the flight-assist system as a form of 
confirmation. 


Thought Commands 


Thought commands are projected to be a faster method to input commands, but face 
the underlying limit of the concentration ability of the human. In the case of the 
breakthrough-era vehicle, the use of thought commands can be considered comparable 
to the use of voice commands. Accordingly, some form of confirmation from the flight- 
assist system is needed, which the pilot must acknowledge before that command is 
implemented. 
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Chapter 4; Future Work 


Given the incomplete body of knowledge concerning futuristic propulsion and 
maintaining cognizance of ongoing human-factors research, it is probably premature to 
engage in specific research on cockpits for propulsion physics. Instead, more insights 
are likely to be gained from relevant physics research. A caveat is that advances in 
cockpits for breakthrough flight might be further advanced by taking advantage of the 
gaming industry or through science fiction speculation. 


Without the actual technology for breakthrough flight, a game simulation or detailed 
science fiction show/movie could be used as the context around which to explore such 
options. A concern with this approach, however, is that the underlying stories serve the 
primary function of entertainment as opposed to user efficiency. It is conceivable to 
encounter a guidance system more intended to create dramatic tension ("wow effect") 
than ease of use. 


MULTIPLE FLIGHT REGIME GUIDANCE CONVENTIONS 


The breakthrough vehicle will operate in regimes for which guidance standards do not 
yet exist, specifically orbit insertion and deep-space (interstellar) travel. Although 
motion near the surface of a gravitating body can copy the standards of aircraft flight 
(primary flight display and terrestrial navigation standards), further work is required to 
explore and select the best options for orbit and deep-space flight. 


Although it is expected that orbit insertion maneuvers would be handled by an 
automated system, the more demanding condition to use as a design target is to have 
a display system that can guide a pilot to manually enter a stable orbit. 


Choosing the convention for the primary axis of deep-space motion will require a trade 
study to determine if the natural human instinct for forward-dominated motion offers a 
better human-machine interface than the propulsion-dominant option for upward- 
dominated motion. Such an assessment must also consider how well the convention 
works when transitioning from deep-space flight into an orbit, then landing, and then 
back again to deep-space flight. Once any convention is set into place, it will be difficult 
to change later. 


VECTOR MOTION DISPLAY 


Once the development of those guidance conventions is further along, a method to 
clearly display those conventions for the pilot would have to be developed. A 
complication that these future displays will encounter is the need for a seamless 
transition between these three conventions: flight above a gravitating body, orbit 
insertion, and deep-space flight beyond our solar system. 


VECTOR MOTION CONTROL 


Adding three linear axes (plus yaw) to the classic two-rotational degree-of-freedom 
joystick is a significant change. Although six-degree joysticks are available 
commercially, they are oriented toward computer interfaces rather than commanding 
the motion of a vehicle. To determine the optimum configuration for an actual six- 
degree vehicle control, simulations would likely be required. Perhaps one venue is to 
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explore the vehicle controls for exploration submarines that also have the full six 
degrees of freedom. 


Another critical detail to be resolved with commanding a breakthrough vehicle is how 
best to accommodate the "throttle." Should each linear axis have the same speed 
capability, or should only the primary motion axis have the full span of zero to 
maximum speed? With that characteristic specified, it will also be necessary to 
determine how best to provide adjustments to cover the speed ranges of each axis. 


OPTIMUM MIX OF CONTROL METHODS 


A variety of methods have been considered for commanding the breakthrough-era 
vehicle. What is not known is the degree and in what combinations they should be 
used, and whether constraints should be placed on some of them. The potential use of 
confirmation feedback for both voice and thought commands is suggested, but future 
research will need to determine whether (and how) such functions should be employed. 
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Appendix A: Annotated Bibliography 


References used in the course of this report are provided here along with a short 
description of each. Although not all were explicitly cited, each influenced this study. 


2 


30 Great Examples of Data Visualization. (2009, June). Web Designer Depot. Retrieved from 
<http://www. webdesignerdepot.com/2009/06/50-great-examples-of-data-visualization> 


Article with examples and links to dozens of different ways to visually convey the interrelationships of 
multiple items - visualization techniques suitable for future gesture-based interfaces and/or art. 


Ausubel, J. & Marchetti, C. (2001). The Evolution of Transport. The Industrial Physicist, April/May, 20-24. 
Article (5 pg) gives historic data on human travel instincts compared to transportation technology — with 
projections aimed to advocate Maglev trains. The data appears constant over history that people average 
1 hr/day of travel, typically round trips from home (80% of the time) and where school and work are the 
temporary destinations. Also, and again consistent over time, the resources devoted to travel is 12-15% 
of total resources. This means for walking (2.5 km/h), the village territory is 20 km?. Driving (40km/h), 
territory is 1,000 km?. For flying (600km/h), the territory was unspecified due to differences in the 
proportion of population using air flight and the differences in time devoted (different travel reasons). 


Bass, L. & Dewan, P. (Eds.). (1993). Trends in Software, User Interface Software, New York, NY: John Wiley 
& Sons. 


Book (216 pg) Addresses usability attributes and then how these are tested and improved. 


Cohen, A. & Chen, E. (1999). Six Degree-of-Freedam HAPTIC System as a Desktop Virtual Prototyping 
Interface, Cambridge, MA: SensAble Technologies, Retrieved from 
<http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.1783&rep=rep1&type=pdf> 
Technical Advertisement (2 pg) Mechanical version of a six-deg-interface that offers farce-feedback oan all 
6 motions. Although not adaptable to cockpit joysticks, the issue of tactile feedback is relevant. 


Coombes, L. F. E. (2005). Contro/ in the Sky: The Evolution and History of The Aircraft Cockpit, Pen & Sword 
Aviation: Leo Cooper Limited. 


Book (320 pg) A historical overview of the developments of the cockpit. Contains significant examples of 
contemporary cockpit displays. 


Degani, A. & Wiener, E. (1990). Human Factors of Flight-Deck Checklists: The Normal Checklist, NASA CR- 
177549, University of Miami, Coral Gables, FL. 


Report (67 pq) Focuses on aircraft checklists and presents recommendations for improvements. 


Dismukes, R. (1991), Aerospace human factors research division - Code FL. Internal document, NASA Ames 
Research Center, Moffett Field, CA. 


Report (83 pg) listing the projects and personnel of that division during that year. 


Elim, W. C. & Waods, D. D. (1985). Getting Lost: A Case Study in Interface Design. In Proceedings: Twenty- 
Ninth Annual Meeting of the Human Factor Society, 927-931, 


Report (5 pg) Applies principles of spatial data management to solve the ‘getting lost’ and 'mode error’ 
phenomena frequently encountered with multi-display networks. 


Genik IT, R. J. (2009/ March/ 23). Technological Approaches to Controlling External Devices in the Absence 

of Limb-operated Interfaces, Defense Intelligence Reference Document, DIA-08-1003-012 (unclassified), 
Report (31 pg) summaries the variety of approaches for having the brain directly control machines, called, 
Brain-Machine Interface (BMI), without the need for mechanical actions. While long-term projections 
suggest that implants might ultimately become viable, the most likely near-term embodiments are 
noninvasive means, such as the existing commercial products MindSet, by NeuroSky, and nia, by OCZ 
Technofogy. Accuracy is improving as this technology is further researched. 
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Grifantini, K. (2010). GM Develops Augmented Reality Windshield. Technology Review, MIT, March 17. 
Retrieved from <http://www.technologyreview.com/blog/editors/24936/?nlid=28288a=f> 


News note on an advanced version or next generation of heads-up display to outline the road, pinpoint 
obstacles (people and signs), even in bad weather. Pertinent excerpt: “To turn the entire windshield into a 
transparent display, GM uses a special type of glass coated with red-emitting and blue-emitting 
phosphors--a clear synthetic material that glows when it is excited by ultraviolet light. The phosphor 
display, created by SuperImaging, is activated by tiny, ultraviolet lasers bouncing off mirrors bundled near 
the windshield. Three cameras track a driver's head and eyes to determine where she is looking." 


Hall, Kenji. (2007). The Big Ideas Behind Nintendo's Wii. Business Week. Retrieved fram 
<http://www.businessweek.com/technology/cantent/nov2006/tc20061116_750580.htm> 
News note on the development of the Wii gaming interface which is a device-dependant gesture-based 
interface. 


Hoover, G. (1992). Stop Saturating Pilots with Alphanumeric Hieroglyphics. Aviation Week and Space 
Technology, June. 15, 98-99. 


Editorial (2 pg) Asserting opinions by an accomplished researcher on what pilots need versus what flight 
instrument designers were providing at that time. In short, with the advent of camputer capabilities, many 
displays were simply providing data in numeric form rather than in some form of interpretable graphic 
format: "Nature does not use symbols, letters or numbers to convey information. As a result, the human 
being has evolved as an effective differentiator, but a poor integrator.” 


Joels, Kennedy, & Larkin. (1982). The Space Shuttie Operator's Manuai/, New York, NY: Ballantine Books, 


Book (179 pg) Public oriented book illustrating the displays and controls of the Space Shuttle, which itself 
is tightly coupled to the legacy aircraft methods. 


Johannesen, L. & Woods, D. D. (1991). Human Interaction with Intelligent Systems: Trends, Problems and 
New Directions, In Conference Proceedings 1991 IEEE International Conference on Systems, Man, and 
Cybernetics, 1337-1341. 


Article (5 pg) Examines lessons and trends and emphasizing the need to have a visual model for the 
process being monitored, 


McCandless, David. (2009). The Visuat Miscellaneum. New York, NY: HarperCollins. 


Book (255 pg) Illustrating numerous ways to graphically convey complex information in readily 
interpretable forms. 


Millis, M. & Davis, E. (Eds.). (2009). Frontiers of Propuision Science, Volume 227 of Progress in Astronautics 
and Aeronautics, Reston, VA: American Institute of Aeronautics and Astronautics (AIAA). 


Book (739 pg) First scholarly textbook about the physics issues, progress, and next-step research 
regarding such game-changing advances as control of gravitation or inertia, propellant-less space drives, 
faster-than-light travel, and related energy requirements. Includes physics regarding the quantum 
vacuum, space-time warping, quantum non-locality, cosmology, and others. Includes chapters to clearly 
document null-results of commonly recurring proposals, and a chapter about how to conduct such 
revolutionary research in contemporary circumstances. 


Moravec H. (2000). Robot: Mere Machine to Transcendent Mind. Oxford University Press. 


Book (240 pg)‘ Fitting introduction to the notion of transhumanism, plus other predictions about the ascent 
of artificial life. This is an updated version of his earlier seminal book, Mind Children: The Future of Robot 
and Human Intelligence. 


Morgan, B. (1985). DOD enlists voice control and expert systems. Efectronic Products, Oct. 1, 65-67. 


Article (3 pg) Relevant in reflecting the early trends to actually apply voice control in military cockpits, 
citing Wright-Patterson AFB as a leading center. 


Moskowitz, S. & Devereux, W. (1970). Navigational Aspect of Transstellar Space Flight. In F. Ordway III 
(Ed.}, Advances in Space Science and Technology. Academic Press, Vol.10, 75-126. 
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Excerpt (15 pg) NASA and Kallsman Instrument Corporation (Syosset, NY) discuss how stellar charts 
change over distances traveled and address Doppler shifts. 


Oran, Daniel. (1991). Apply Psychology to Product Design. Machine Design. Nov. 7, 37-40. 


Article (4 pg) Illustrates how to use different switch locations, groupings, and shapes to more readily 
convey functions. 


Paviovic', V. I., Sharma, R. & Huang, T. S. (1997). Visual Interpretation of Hand Gestures for Human- 
Computer Interaction: A Review. In JEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 
677-695 


Article (36 pgs) Summarizes recent progress toward using gestures of the arms and hands to interact with 
computers. Hand gestures, including those with different finger positions, can be used. These features 
are likely to mature soon into commercial products. 


Russo, D., Tillman, B., & Pickett, L. (2009). Human Factors Engineering Standards at NASA, Presented at 
the Human Factors and Ergonomics Society Conference (22-26 Sep. 2008, Manhattan, NY}. NASA Johnson 
Space Center, Houston TX. 


Quoting Abstract: "NASA has begun a new approach to human factors design standards, For years NASA- 
STD-3000, Manned Systems Integration Standards, has been a source of human factors design guidance 
for space systems. In order ta better meet the needs of the system developers, NASA is revising its 
human factors standards system. NASA-STD-3000 will be replaced by two documents: set of broad human 
systems design standards (including both human factors and medical topics} and a human factors design 
handbook. At the present time the standards document is in final review with some disagreement on 
several critical issues. The handbook {s progressing with November 2008 as the anticipated completion 
date." 


Potter, S. S. & Woods, D, D. (1991). Event Driven Timeline Displays: Beyond Message Lists in Human- 
Intelligent System Interaction. In Conference Proceedings 1991 IEEE International Conference on Systems, 
Man, and Cybernetics, 1283-1288. 


Article (6 pg) Examines lessons and trends of fault monitoring, suggesting that timelines of events be 
visually conveyed, including causal connections. 


Patter, S. S. et al. (1992), Visualization of Dynamic Processes: Function-Based Displays for Human- 
Intelligent System Interaction. In Proceedings 1992 IEEE International Conference on Systems, Man, and 
Cybernetics, 1283~1288. 


Article (6 pg) Although in the context of monitoring processes, it suggests that visual models of the 
processes (instead of alphanumeric displays) are crucial. 


Proffitt, D. & Kauiser, M. (1991). Observer Properties for Understanding Dynamical Displays: Capacities, 
Limitations, and Defauits. NASA TM 102812. Ames Research Center, Moffett Field, CA. 


Report (15 pg) Examined human ability to understand and predict motion in the display of dynamic 
systems. They found that both physics-educated people and naive people perform about equally on the 
intuitive level: particle motion is relatively accurate, but rotational kinematics, especially that concerning 
moments of Inertia, Is relatively poor, 


Reason, J. (1990). Human Error. Cambridge, UK: Cambridge University Press. 


Book (316 pg) By examining major accidents, the correlation between psychology and the operation of 
hazardous technologies is described. As such, it presents a wealth of lessons of mistakes to avoid. 


marter, N. & Woods, D. (1992). Mode error in supervisory control of automated systems. In Proceeding of 
the Human Factors Society 36'" Annual Meeting. Atlanta GA, Oct. 


Report (5 pg) Illustrates and defines mode error: "When an action entirely appropriate for a situation is 
performed, except that this is not the current situation.” -Norman 1981. Describes how systems that 
have command sequences, which change their pattern in different operating modes, is a problem. 


Seifert, H. (Ed). (1959), Space Technology. New York, NY: Wiley & Sons. 
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Book (several hundred pages) Early scholarly textbook about space technology in general, including 
Interesting projections on displays and controls [Figure 29-16]. Despite the age of this text, much of Its 
foundational coverage is still valid. 


Stanton, N., et al. (2005). Human Factors Methods; A Practical Guide For Engineering and Design. Aldershot, 
Hampshire: Ashgate Publishing Limited. 


Book (571 pg) Title says it all. 


Shneiderman, B. (1997). Designing the User Interface: Strategies for Effective Human-Computer Interaction, 
3rd edition. Boston, MA: Addison-Wesley Longman Publishing. 


Book (639 pg) Covers a variety investigations, in particular the “Visual Infermation Seeking" (VIS) 
principles by which people look for information. Like so many other references, it discusses how people 
use visual models either directly or indirectly analogous to physical objects to arrange information. Also 
goes Into detail about 


ec Time to learn 

c Retention time 

c Rate of errors 

ce Error recovery 

e Cultural and individual differences 


Stokes, A., Wickens, C. & Kite, K. (1990). Display Technology - Human Factors Concepts. Warrendale, PA: 
Society of Automotive Engineers, Inc, 


Book (167 pgs) This was a major source of information for this study. Funded by General Motors, this is a 
compilations of a variety of display research results, focusing on systems for pilots and drivers. 


Sutter, J. 0. (2010/ June/ 18). New tech moves beyond the mouse, keyboard and screen. CNW, Retrieved 
fram Fae 
<http://www.cnn.com/2010/TECH/innovation/06/18/natural.user.interface. mouse/index.html?hpt=C2>. 


News article describing a number of emerging technologies for the computer interface, including voice, 
gesture commands, brain-controlled computing, eye-tracking software, and even having the images 
appear virtually around you via special contact lenses. Cites the new Microsoft Kinect gesture-based 
gaming system, and John Underkoffler - the science advisor to the 2002 movie, Minority Report, where 
provocative interactions between the user and data are shown. Also reminds us that voice commands and 
touch screens are now cammon with cell phones. Regardless of such advances, the keyboard is still likely 
to be around for those lingering tasks that still cannot be comfortably done with the new "Natural 
Interface” technology. Also discusses the difficulty with such systems, citing the tiring aspect of gesture- 
based systems, the likelihood of misinterpretation of motions, and the lack of tactile feedback. 


Tufte, E. (1991). Envisioning Information. Cheshire, CT: Graphics Press. 


Book (126 pg) Comprehensive examples about graphically presenting information, including bad examples 
to avoid. Useful for general rules on use of colar, shading, lines, etc. Although focused on static display of 
information (print material instead of dynamic displays), many of the lessons of human perception equally 
apply. It does not address dynamic displays or system integration. 


Tilley, A. R., & Dreyfuss, H, (2001). The Measure of Man and Woman: Human Factors in Design. Wiley. 


Book (104 pg) Comprehensive information regarding anthropometric charts - dimensions and positions of 
humans, male and female, to be used as references when designing chairs, consoles, or other human 
facters equipment. This builds on decades of work from designer Henry Dreyfuss who first started 
publishing in this field in 1955. 


Underkoffler, J. (2010). John Underkoffler Points to the Future of User Interfaces. TED Ta/ks, June 2, 2010, 
Retrieved from <http://www.huffingtonpost.com/tedtalks/john-underkoffler-points_b_597501.html> > 


Lecture (YouTube) About the status and projections of gesture-based user interfaces. Underkoffler was 
the science advisor to the 2002 movie, Minority Report, where provocative interactions between a user 
and data are shown. He is also a 15-year veteran of MIT's Media Laboratory. The key take-away points 
from this lecture include projections that gesture-based interfaces will be available in products before 
2015, and will no longer require gloves or other devices to be worn by the user. Errors, however, such as 
the computer misinterpreting an unintended motion as a command input, are likely to remain issues. 
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Although not mentioned in the lecture (advertising is not allowed at TED talks), Underkoffler's Obfang 
Industries |s already working to make those systems real. 


Van der Vaart, A. J. M. (1995). Arm movements in operating rotary controls (Doctoral dissertation). 
Techniche Universiteit Delft, Netherlands, Delftse Univ. Pres. 


Dissertation (169 pg) Intense detail into arm and hand motions when dealing with controls such as knobs 
and joysticks - including experimental test methods. Sections both in English and Dutch. Many useful 
diagrams for those designing the actual mechanics of controls. Sultable information if designing a new 
six-degree of freedom joystick. 


Visual Understanding Environment (VUE). (2008). Tufts University. Site developed by Academic Technology. 
Retrieved from <http://vue.tufts.edu/> 


Downloadable, open source software to create flexible tools for managing and integrating digital resources 


In support of teaching, learning and research. "VUE provides a flexible visual environment for structuring, 
presenting, and sharing digital information." 


Wan, C, et al. (1991). "Visé: A Visual Programming Environment for Data Analysis and Visualization." 
Presented at the 1999/Nov. Conference of Association for Computing Machinery (ACM) User Interface 
Software and Technology (National Information Display Laboratory), 
Report (7 pg) Pertinent in the context of reflecting the degree of difficulty when presenting information 
about the health of an electromechanical system, where comprehending and reacting to the data is not 
straightforward, Hierarchical organization covered, 


Webb, P. (Ed.). (1964), SECTION 17. VISION, Bioastronautics Data Book. NASA SP-3006. Washington DC: 
Scientific and Technical Information Division. 


Section from Book containing data on human vision, field of view, effects of lighting, ete. 


Woods, D. D, (1984), Visual momentum: a concept to improve the cognitive coupling of person and 
computer. International Journal of Man-Machine Studies, 21, 229-244. 
Report (16 pg), Examines the problem of getting lost when working with the multiple display windows that 
are now common in computer displays. The taxonomy is also useful. 


Woods, D. D. (1988). The Significance Messages Concept for Intelligent Adaptive Data Display. Working 
Paper Series No. 1988-015. Columbus OH: Ohio State University. 
Report (21 pg) Discusses how intelligent monitoring systems can take on the function of combining a 
number of variables to calculated a condensed and meaningful "Significant Message” to the user, For 
example, instead of just displaying various states of a system, the intelligent system takes all those values 
to give the user a key message, such as "Pump 1 at reduced capacity." 


Woods, D. D. (1993). The price of flexibility. In Hefley & Murray (Eds.), Proceeding of the International 
Workshop on Intelligent User Interfaces. Association for Computing Machinery (ACM). 


Report (7 pg) Illustrates how adding flexibility to information displays increased the burden on the users. 
Users dealt with this by setting a default fixed display and by avoiding using the systems during high- 
stress moments. A list of corrective measures is provided for designing intelligent user interfaces. 
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